Image display apparatus for shovel

ABSTRACT

An image display apparatus for a shovel that includes a lower-part traveling body, an upper-part turning body turnably placed on the lower-part traveling body, and an attachment mounted on the upper-part turning body, and performs work in a work area by driving the attachment, includes an image display part configured to display the work area to be subjected to the work. An image of the work area captured with an image capturing device is subjected to image processing to add and indicate a distribution of depressions and elevations of the work area, and is displayed on the image display part.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 14/301,419, filed on Jun. 11, 2014, which is a continuationapplication filed under 35 U.S.C. 111(a) claiming benefit under 35U.S.C. 120 and 365(c) of PCT International Application No.PCT/JP2012/080517, filed on Nov. 26, 2012 and designating the U.S.,which claims priority to Japanese Patent Application No. 2011-284095,filed on Dec. 26, 2011. The entire contents of the foregoingapplications are incorporated herein by reference.

BACKGROUND Technical Field

The present invention relates to image display apparatuses that displayan image of a work area of a shovel.

Description of Related Art

In general, an operator of a construction machine such as a shoveloperates a work element of a shovel, sitting on an operator seat in acabin and looking at a work site. The typical work of shovels is, forexample, digging earth and sand and loading the bed of a truck with thedug-out earth and sand. In this case, first, the operator digs earth andsand with a bucket while looking at an excavation site. Thereafter, theoperator moves the bucket to above the bed of the truck, and dumps outthe earth and sand inside the bucket onto the truck while looking at theearth and sand loaded onto the bed of the truck.

In the above-described work, first, the operator looks at the conditionof the earth and sand of the excavation site and determines a part to bedug with the bucket. Normally, the operator determines the part to bedug by the depressions and elevations (undulations) of the excavationsite. In the case of digging a hole in the excavation site, the operatorfurther digs a depressed part for a deeper hole. In the case offlattening the excavation site, the operator flattens out projectingparts. Thus, shovel operators often perform excavation work whiledetermining the distribution of depressions and elevations of anexcavation site.

Furthermore, in the work of loading the bed of the truck with earth andsand, the operator dumps out earth and sand onto part of the bed of thetruck that has been loaded with less earth and sand. That is, theoperator performs the work of dumping out earth and sand onto a part ofthe earth and sand that is lower in level while determining thedistribution of depressions and elevations of the earth and sand loadedon the bed of the truck.

Shovel operators perform work, successively determining the conditionsof a work site as described above, and are therefore required torecognize the conditions of the work site all the time.

Here, a shovel has been proposed that displays an image captured with acamera provided on the counterweight of a shovel on a monitor in anoperator room and provides the operator with an image of the backside ordiagonal backside that is blind to the operator at the time ofoperation.

SUMMARY

According to an embodiment of the present invention, an image displayapparatus for a shovel that includes a lower-part traveling body, anupper-part turning body turnably placed on the lower-part travelingbody, and an attachment mounted on the upper-part turning body, andperforms work in a work area by driving the attachment, includes animage display part configured to display the work area to be subjectedto the work. An image of the work area captured with an image capturingdevice is subjected to image processing to add and indicate adistribution of depressions and elevations of the work area, and isdisplayed on the image display part.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and notrestrictive of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side view of a shovel provided with an image displayapparatus according to an embodiment of the present invention;

FIG. 2 is a side view of a shovel performing the work of loading a bedof a truck with the earth and sand of a bucket;

FIG. 3 is a diagram illustrating an image displayed by the image displayapparatus according to the embodiment of the present invention; and

FIG. 4 is a block diagram illustrating an overall configuration of theimage display apparatus.

DETAILED DESCRIPTION

Sight from the operator seat of the shovel is limited. For example, ifthere are large depressions and elevations in an excavation site, it isimpossible for the operator to visually recognize how it is shapedbeyond an elevation. Furthermore, in the case where a work area is abovethe operator as in the case of the bed of a truck, the operator looks upat the bed. Therefore, the operator is prevented from having a view ofthe entire inside of the bed, and is thus prevented from recognizing thecondition of earth and sand loaded onto the bed.

Furthermore, an image is captured with the camera and displayed on themonitor only to the extent that the outline of a displayed object may bedetermined. The operator of the shovel is not performing work, lookingonly at the monitor. Furthermore, the monitor screen is extremely smallcompared with a visual field. Accordingly, the size of an objectdisplayed on the monitor screen is much smaller than in an actual view.For these reasons, the operator of the shovel is prevented frominstantaneously recognizing, for example, a distribution of finedepressions and elevations in a work area from an image displayed on themonitor.

Thus, performing work in a situation where an operator is prevented fromhaving a good understanding of the condition of a work area results in,for example, an error in the excavation position. Furthermore, an errormay be caused in the position at which the bed of a truck is loaded withearth and sand. Accordingly, the work efficiency of the shovel may bereduced by extra work for correcting such positional errors orre-performing work.

According to an aspect of the present invention, an image displayapparatus for a shovel is provided that enables a shovel operator toinstantaneously recognize the conditions of a work area through an imagedisplayed on a monitor.

According to an aspect of the present invention, the distribution ofdepressions and elevations in a work area is displayed in an image.Therefore, even in the case where it is impossible or difficult to viewthe conditions of the work area, it is possible for an operator toinstantaneously recognize a position at which work is to be performedand to perform work at an exact position. As a result, it is possible toimprove the work efficiency of the shovel.

A description is given below, based on the drawings, of an embodiment ofthe present invention.

FIG. 1 is a side view of a shovel provided with an image displayapparatus according to an embodiment of the present invention,illustrating a state during excavation work.

The shovel includes a lower-part traveling body as a drive part fortraveling. An upper-part turning body 3 is mounted on the lower-parttraveling body 1 through a turning mechanism 2. A boom 4 is attached tothe upper-part turning body 3. An arm 5 is attached to an end of theboom 4, and a bucket 6 is attached to an end of the arm 5. The boom 4,the arm 5, and the bucket 6 are hydraulically driven by a boom cylinder7, an arm cylinder 8, and a bucket cylinder 9, respectively. The boom 4,the arm 5, and the bucket 6 are configured as attachments of the shovel.Work operations such as excavation and flattening in a work area areperformed by driving the attachments.

A cabin 10 is provided on the upper-part turning body 3 as an operatorroom in which an operator seat and an operation apparatus are placed.Power sources such as an engine, hydraulic pumps, and a hydrauliccircuit are mounted behind the cabin 10 on the upper-part turning body3.

The configuration of a drive system of the shovel is the same as that ofdrive systems of common shovels, and its description is omitted.

FIG. 1 illustrates a state where the shovel is performing excavationwork. In the excavation work, some part of a work area, such as a part Ain FIG. 1, is not visible from an operator because of depressions andelevations of the work area. Therefore, in this embodiment, a camera 20is provided at an end of the arm 5 of the shovel, an image of the workarea is captured with the camera 20, and the obtained image is displayedon a monitor (an image display part) 30 at the operator seat.

Because an image of the work area may be captured from directly abovethe work area with the camera 20, it is possible to capture an image ofa part (the part A in FIG. 1) that is not visible to the operator.Accordingly, it is possible for the operator to check the conditions ofthe entire work area by looking at the screen of the monitor 30.

Images captured with the camera 20, however, are not very sharp, and themonitor 30 is limited in size. Accordingly, the operator cannotimmediately understand the conditions of depressions and elevations,such as which part is low and which part is high, by looking at a workarea displayed on the monitor 30. Therefore, according to an imagedisplay apparatus of this embodiment, an image captured with the camera20 is displayed on the monitor 30 with the addition of an imageindicating depressions and elevations in a region inside the image. Theimage indicating depressions and elevations is, for example, contourlines that indicate parts of the same height in the image, and is animage displayed in such a manner as to indicate mountains and valleys incommon maps so as to facilitate an understanding of depressions andelevations in a planar image. A detailed description is given of theimage indicating depressions and elevations in relation to the loadingof earth and sand illustrated in FIG. 2.

FIG. 2 is a side view illustrating a state where the earth and sandscooped up with the bucket 6 is being loaded onto the bed of a truck.The earth and sand needs to be loaded onto a bed 40 a of a truck 40 tobe uniform in height. In the situation illustrated in FIG. 2, however,the operator would dump out the earth and sand inside the bucket 6 ontoan approximate position because the operator is prevented from lookinginside the bed 40 a of the truck 40.

Therefore, according to this embodiment, an image of the bed 40 a of thetruck 40 captured with the camera 20 is displayed on the monitor 30 inthe operator room of the shovel. At this point, the image captured withthe camera 20 is displayed with the addition of contour lines indicatingdepressions and elevations as illustrated in FIG. 3, so as to enable theoperator to immediately recognize depressions and elevations in the workarea.

FIG. 3 is a diagram illustrating a screen of the monitor 30 at the timewhen an image of the bed 40 a of the truck 40 captured with the camera20 is displayed on the monitor 30 with the addition of contour lines(dotted lines) thereto. On the screen of the monitor 30, the entire bed40 a of the truck 40 is displayed, and contour lines 50-1 through 50-4are displayed so that the undulations of the earth and sand inside thebed 40 a may be recognized.

In the loading work, it is necessary to load earth and sand onto theentire bed 40 a as uniformly as possible by dumping out earth and sandonto parts that are lower in level (depressed parts). Therefore, theoperator of the shovel determines the undulations (depressions andelevations) of the earth and sand on the bed 40 a based on the contourlines 50-1 through 50-4 displayed on the screen of the monitor 30 wherethe bed 40 a of the truck 40 is displayed, and dumps out the earth andsand in the bucket 6 onto lower parts. Even in the case where it isdifficult to recognize undulations (depressions and elevations) with thescreen of the monitor 30 alone, it is possible to immediately determineundulations (depressions and elevations) by looking at the contours 50-1through 50-4, so that the work efficiency is improved. Furthermore,there is no need to perform extra work such as discharging earth andsand further onto higher parts and flattening them afterward, so that itis possible to further improve the efficiency of work by the shovel.

In the case illustrated in FIG. 3, of the contour lines 50-1 through50-4, the contour lines 50-1 are lines that connect the lowest parts,the contour lines 50-2 are lines that connect the next lowest parts, thecontour lines 50-3 are lines that connect the third lowest parts, andthe contour line 50-4 is a line that connects the highest parts. Theinterval (difference in height) between contour lines may be set to anappropriate value, for example, 30 cm. Alternatively, the intervalbetween contour lines may be set to a predetermined value in advance orbe set by inputting a value that facilitates the operator'sunderstanding.

Furthermore, the contour lines 50-1 through 50-4 may be assigneddifferent colors. For example, the contour lines 50-1 that indicate thelowest positions may be in blue, the contour line 50-4 that indicatesthe highest positions may be in red, and the intervening contour lines50-2 and 50-3 may be in appropriate colors such as those graduallychanging from blue to red. Alternatively, the entire image may besubjected to coloring or hatching with the same color or hatching beingapplied to a part of the same height (the region between adjacentcontour lines), so that undulations (depressions and elevations) may begradually displayed in the entire image.

A determination as to undulations (depressions and elevations) in thework area may be made by the image display apparatus processing an imagein the work area and performing image recognition. Alternatively, theundulations may be determined by measuring the distances betweenmultiple positions in the work area and the camera 20 by incorporating adistance sensor in the camera 20. The distance sensor may be, forexample, a reflective distance sensor.

Furthermore, in the case illustrated in FIG. 3, an instruction areaframe 60 that indicates a work instruction area is further displayed inaddition to the contour lines 50-1 through 50-4 on the screen of themonitor 30. The area surrounded by the instruction area frame 60 is thework instruction area. In the case illustrated in FIG. 3, theinstruction area frame 60 is an area loaded with less earth and sand inthe bed 40 a, and is an area yet to be loaded with earth and sand. Theimage display apparatus, based on the contents of work, specifies anarea to be worked on (that is, a work instruction area) in the image ofa work area displayed on the monitor 30 and displays the area to beworked on by surrounding the area to be worked on with the instructionarea frame 60 on the screen. In the case illustrated in FIG. 3, becausethe work is the loading of earth and sand, a part of the earth and sandthat is lower in level is displayed as the work instruction areasurrounded by the instruction area frame 60, so as to make the height ofthe earth and sand as uniform as possible.

When the work is, for example, excavation, an area to be excavated isdisplayed as the work instruction area surrounded by the instructionarea frame 60. Furthermore, when the work is to flatten out earth andsand, a part of the earth and sand that is higher in level (an elevatedpart) is displayed as the work instruction area surrounded by theinstruction area frame 60.

The display of the work instruction area is not limited to surrounding awork instruction area with the instruction area frame 60 as illustratedin FIG. 3. For example, the work instruction area may be displayed byperforming such image processing as to make the work instruction areadistinguishable from other areas, such as coloring or hatching the workinstruction area.

An image displayed on the monitor 30 is of the entire work area in whichexcavation or loading is performed with the shovel. It is also possibleto display the entire work area inside the image on the monitor 30 byregistering the position of the work area as a coordinate position inadvance. In FIG. 3, the work area is a part surrounded by a dotted lineframe 70. The coordinate position of the work area may be detected alongwith the coordinate position of the shovel using a GPS function. In thiscase, a GPS device is incorporated into the camera 20. If possible,undulations (depressions and elevations) in the work area may also bedetermined by the GPS function.

FIG. 4 is a system configuration diagram of an image display apparatus100 according to this embodiment. The image display apparatus 100includes a control part 102 that includes an image processing part 110,a signal processing part 120, a coordinate operation part 130, and awork area determination part 140.

The image data of a work site captured with the camera 20 aretransmitted to the image processing part 110. The image processing part110 processes the image data into image data to be displayed on themonitor 30. The image data processed in the image processing part 110are transmitted to the work area determination part 140.

In the case where a GPS device 22 is incorporated in the camera 20, theposition coordinate data of the camera 20 detected by the GPS functionare transmitted to the work area determination part 140. The positioncoordinate data are fed to the coordinate operation part 130 as well.

Furthermore, in the case where a distance sensor 24 is incorporated inthe camera 20, distance data are transmitted to the signal processingpart 120. The distance data are data that represent the distance fromthe camera 20 to each of positions in the work area. The distance dataprocessed in the signal processing part 120 are fed to the coordinateoperation part 130. The coordinate operation part 130 creates theposition coordinate data of an area with respect to which the distancehas been measured by correlating the positions measured by the distancesensor 24 with the position coordinate data, based on the positioncoordinate data from the GPS device 22 and the distance data from thedistance sensor 24, and outputs the position coordinate data to the workarea determination part 140.

The work area determination part 140 creates an image to be displayed onthe monitor 30 using the image data from the image processing part 110,the position coordinate data from the GPS device 22, and the positioncoordinate data from the coordinate operation part 130, and transmitsmonitor image data to the monitor 30. The work area determination part140 recognizes undulations (depressions and elevations) in the imagefrom the image data from the camera 20, and adds data representing thecontour lines 50-1 through 50-4 to the image data. Alternatively, in thecase where the data representing undulations (depressions andelevations) (distance data) are fed from the distance sensor 24, thedata representing the contour lines 50-1 through 50-4 may be createdbased on the position coordinate data fed from the coordinate operationpart 130.

Furthermore, the work area determination part 140 specifies a workinstruction area to be worked on inside the dotted line frame 70indicating a work area based on the data representing differences inheight, and generates data indicating the instruction area frame 60. Asdescribed above, in the case of the loading of earth and sand (thisinformation is input in advance by the operator), the instruction areaframe 60 is formed so as to surround a part that is lower in level inthe work area.

The image data created in the work area determination part 140 aretransmitted to the monitor 30 provided in the operator room of theshovel and are displayed on the screen of the monitor 30. It is possiblefor the operator to efficiently perform work by performing work, viewingthe contour lines and the work instruction area on the screen of themonitor 30.

Here, in the case where the control part 102 of the image displayapparatus 100 is provided in the shovel, the data from the camera 20,the GPS device 22, and the distance sensor 24 are transmitted to thecontrol part 102 via interconnects inside the shovel. Furthermore, theimage data to be displayed on the monitor 30 also are transmitted to themonitor 30 via interconnects inside the shovel. Each of the camera 20,the GPS device 22, the distance sensor 24, and the monitor 30 does notnecessarily have to be connected to the control part 102 viainterconnects, and the data may be transmitted by radio transmission.The above-described embodiment, of which a description is given, takingloading work as an example, is also effective for flattening work thatrequires formation of a horizontal surface.

Employment of radio transmission makes it possible to install thecontrol part 102 and the monitor 30 in a location distant from theshovel. For example, in the case of performing work by remotelyoperating the shovel, a remote operator is often prevented from having aview of a work area. Even in such a case, by remotely operating theshovel while displaying an image based on the image data generated bythe image display apparatus according to this embodiment, it is possibleto efficiently accomplish work. Furthermore, the monitor (image displaypart) 30 according to this embodiment may be implemented by a screen ofa portable information terminal such as a smartphone.

Embodiments of the present invention may be applied to an image displayapparatus that displays an image of a work area of a shovel.

All examples and conditional language provided herein are intended forpedagogical purposes of aiding the reader in understanding the inventionand the concepts contributed by the inventor to further the art, and arenot to be construed as limitations to such specifically recited examplesand conditions, nor does the organization of such examples in thespecification relate to a showing of the superiority or inferiority ofthe invention. Although one or more embodiments of the present inventionhave been described in detail, it should be understood that the variouschanges, substitutions, and alterations could be made hereto withoutdeparting from the spirit and scope of the invention.

What is claimed is:
 1. A shovel comprising: a lower-part traveling body;an upper-part turning body turnably placed on the lower-part travelingbody, the upper-part turning body including an attachment to be drivento perform a work operation of the shovel; an image display apparatus ina cabin provided on the upper-part turning body; a camera provided onthe upper-part turning body; and a distance sensor provided on theupper-part turning body.
 2. The shovel as claimed in claim 1, furthercomprising: a controller configured to receive image data captured withthe camera and to display an image on the image display apparatus basedon the received image data.
 3. The shovel as claimed in claim 2, whereinthe controller is further configured to receive distance data from thedistance sensor and to display the image on the image display apparatusbased on the received image data and the received distance data.
 4. Theshovel as claimed in claim 3, wherein the controller is furtherconfigured to create data representing an elevation and a depression ofa work area based on the received distance data.
 5. The shovel asclaimed in claim 3, wherein the controller is further configured togenerate an instruction area frame indicating a work instruction areabased on the received distance data.
 6. The shovel as claimed in claim5, wherein the instruction area frame is specified based on contents ofwork.
 7. The shovel as claimed in claim 3, wherein the controller isfurther configured to create position coordinate data based on thereceived distance data.
 8. The shovel as claimed in claim 7, wherein thecontroller is further configured to display a frame on the image displayapparatus based on the created position coordinate data.
 9. The shovelas claimed in claim 7, wherein the controller is further configured todisplay the image on the image display apparatus with information of thecreated position coordinate data being placed over the image.
 10. Animage display method in a shovel, the shovel including a lower-parttraveling body, an upper-part turning body turnably placed on thelower-part traveling body and including an attachment to be driven toperform a work operation of the shovel, an image display apparatus in acabin provided on the upper-part turning body, a camera provided on theupper-part turning body, a distance sensor provided on the upper-partturning body, and a controller, the image display method comprising:receiving, by the controller, image data captured with the camera; anddisplaying, by the controller, an image on the image display apparatusbased on the received image data.
 11. The image display method asclaimed in claim 10, further comprising: receiving, by the controller,distance data from the distance sensor; and displaying, by thecontroller, the image on the image display apparatus based on thereceived image data and the received distance data.
 12. The imagedisplay method as claimed in claim 11, further comprising: creating, bythe controller, data representing an elevation and a depression of awork area based on the received distance data.
 13. The image displaymethod as claimed in claim 11, further comprising: generating, by thecontroller, an instruction area frame indicating a work instruction areabased on the received distance data.
 14. The image display method asclaimed in claim 13, wherein the instruction area frame is specifiedbased on contents of work.
 15. The image display method as claimed inclaim 11, further comprising: creating, by the controller, positioncoordinate data based on the received distance data.
 16. The imagedisplay method as claimed in claim 15, further comprising: displaying,by the controller, a frame on the image display apparatus based on thecreated position coordinate data.
 17. The image display method asclaimed in claim 15, further comprising: displaying, by the controller,the image on the image display apparatus with information of the createdposition coordinate data being placed over the image.